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i.  IHTBODUCTTOIf 

The  n ew  projective  linear  programming  algorithm  by 
karmarkar  £Pef.  1j  requires  solution  of  svmmetric,  Dositive 
definite  systems  of  equations.  One  of  the  keys  to  solving 
these  systeas  efficiently  is  controlling  the  fill-in,  i.e. , 
the  number  of  new  nonzero  elements  in  the  reduced  matrix,  in 
the  various  forms  of  Gaussian  elimination  used  to  solve 
these  systems.  finder  the  assumption  of  noncancellation  o* 
terns,  fill-in  created  by  direct  solution  techniques  is 
affected  only  by  svmnetric  reordering  *  of  the  rows  and 
columns  of  the  system  to  be  solved. 

The  problem  of  minimizing  fill-in  by  symmetric  reord¬ 
ering  is  known  to  be  HP-complete  fPef.  pi  so,  in  this 
thesis,  we  examine  a  number  of  heuristic  reordering  algo¬ 
rithms  for  this  nroblem. 

Several  different  algorithms  for  reducing  fill-in  in 
Gaussian  elimination  are  compared  for  sixteen  lane-scale 
matrices  derived  from  linear  and  integer  nrooramml no 
oroblems. 


B.  PBOJBCTITB  LP  ALGOBITHU 

Consider  the  linear  programming  problem 
T 

min  c  x 
s.t.  kx  =  b 
x  >  0 


where  A  is  an  m-by-n  matrix,  e  is  an  n-element  colnan  vector 
and  b  is  an  a-eleaent  colnan  vector.  .  Karmarkar’s  algorithm 
requires  the  solution  of  BBz  3  Be,  where 


B  = 


AB  | 


,  B  is  an  n-by-n  diagonal  matrix,  and  1  is 


a  n-eleaent  colnan  vector  of  Is.  Tbns, 


T 

BB  * 


AB*A  AB1 

T  T  i 

1  BA  n 


We  will  not  be  concerned  with  the  bottom  row  and  righ*- 
aost  column;  they  will  tvpicallv  be  100*  dense  when  using  an 
artificial  starting  solution  and  will  be  nlaced  last,  in  nnv 
ordering.  Under  the  assumption  of  noncancellation  of  terms, 
the  nonzeros  created  in  Aa"1"  will  be  the  same  as  in  AB* •T" , 
both  when  creating  these  matrices  and  when  oerformino  the 
elimination.  Thus,  we  may  restrict  study  to  solving  svstems 
of  the  form  AATz  =  Ac  =  b • . 


C.  DIRECT  SOtOTIOR  HBTHODS 

Symmetrv  and  positive  definiteness  are  important  in 
computation  with  the  matrices.  ’’'he  matrix  D  is  symmetric 
when  Dt  *  D.  Otherwise,  n  is  asvmmetric.  A  svmmetric 
matrix  D  is  said  to  be  positive  definite  if  zoz  >  0  for  all 
nonzero  vectors  z. 

Three  basic  methods  are  available  for  direct  solution  of 
positive  definite  systems  of  the  form  AA^z  =  nz  =  h*.  ’’•hese 
methods  are  basic  Gaussian  elimination  r Pef.  f  7,  Cholestv 
factorization  r*ef.  41  and  os  factorization  reef.  "H.  All 
three  methods  end  np  creating  an  upper  or  lower  triangular 
matrix  which  is  used  with  b*,  or  a  modified  version  of  b», 
to  solve  for  z  by  backward  and/or  forward  substitution. 


One  key  to  success  of  these  nethods  is  keeping  the 
triangular  aatrix  as  sparse  as  possible  by  symnetricallv 
reordering  the  rovs  and  colnnns  of  D.  This  keeps  computa¬ 
tion  tine  down  and  ronnd-off  errors  to  a  minimum. 

1.  Basic  Sanssian  Elimination 

The  Gaussian  eliaination  method  we  will  consider  for 
the  solution  of  a  set  of  equations  is  -Just  an  generalization 
of  the  faailiar  method  of  eliminating  one  unknown  by  substi¬ 
tution  between  a  Dair  o*  simultaneous  equations  and  is  the 
basic  pattern  of  a  large  number  of  solution  methods  that  can 
be  classed  as  direct  methods. 

Let  us  consider  the  system  Hz  =  b*,  represented  by 
the  augmented  matrix  [D,b»‘],  where  D  is  an  m-hy-n  uositiv® 
definite  symmetric  matrix  and  b*  is  an  m-vector.  Gaussian 
elimination  by  columns  consists  of  n  steps.  ""he  puroos®  o* 
the  kth  step  is  to  eliminate  all  the  nonzero  elements  of  the 
augmented  matrix  which  lie  on  column  k  below  the  diacronal  of 
P.  kt  the  kth  step,  d<k>  is  the  diagonal  o*  th® 
active  submatrix  {formed  by  rows  k  to  o  and  column  k  to  m  o* 
P<k>)  is  selected  to  be  the  pivot.  Bow  k  o*  the  augmented 
matrix  is  normalized  and  the  elements  in  column  k  below  the 
diagonal  of  p*k»  are  eliminated  by  subtraction  of  multiples 
of  the  normalized  row  k  *roa  all  those  rows  which  hav®  a 
nonzero  in  column  k  below  the  diagonal.  ’’“he  augmented 
system  fp<k>,b«<k>1  is  obtained  where  D<k>  has  zeros  in  i*s 
first  k  columns  below  the  diagonal  and  Is  on  the  *irst  k 
positions  of  the  diagonal. 

The  process  is  continued  until  the  upper  triangular 
system  f  o<  m> ,  b* <m  >  "J  is  obtained  and  z  can  be  solv®d  for  bv 
backward  substitution. 


a 


There  exists  a  nniane  loser  triangular  satrix  t.  such 
that  LLt  =  t>.  rholesfcy  factorization  creates  I  froa  D 
directly  without  consideration  of  b*.  Then,  ItTz  =  =  b' 

is  solved  for  by? 

(1)  Solve  Ly  =  b*  for  y. 

T 

(2)  Solve  L  x  -  y  for  r. 

Suppose  a  positive  definite  symmetric  m-by-m  matrix 
D  is  liven.  The  way  in  which  fill-in  occurs  when  computing 
L  is  best  illustrated  by  showing  a  proof,  following  Ceoroe 
and  Liu  (Ref.  m,  of  one  of  the  rholesby  factorization 
techniques. 

T 

Let  0  *  Id  a  1  where  d  is  a  scalar,  a  is  an  m-1 
1  1  11  1  1 


a  * 

1  1 


vector  and  K,  is  an  (*-1) -by- fm- 1)  subwatrix. 
write 


1/2  T  1/2 

0  l  1  0  Id  a  Ai  l 

I  11  111 

I  1/2  |  1  1 

la  /d  T  \  o  g  io  t  i 

1  1  m-1  1  m-1 


■then  we  can 


1  0  T 

=  T,  L 

1  1 
0  H 

1 


=  L  D  L  , 
1  1  1 


where  g  =  *-  aa/d,  and  I  denotes  an  identity 


D  =  1  T  T,  . 
n-1  m  m  m 

Uere,  for  1<i<m,  d.£  is  a  positive  scalar,  a*  is  a 

vector  of  length  a-i,  and  ?•  is  an  (m-ij  -bv-  (m-ii  oositive 
definite  symmetric  matrix. 

After  m  steps  o*  *he  algorithm,  ve  have 

0  =  L1  ...  L  L.  .  .  LL  =  T I  ‘ 

12  mm  2  1 


where  i*  can  he  showr  that 


I.  =  1  *  L  -  fa-l)T  . 

12  m  m 


Thus,  the  ith  column  of  L  is  oreciselv  th°  ith  column  of  Tx. 

Tn  this  scheme,  the  columns  of  L  are  commuted  one  *v 
one.  At  the  sane  time,  each  step  involves  the  modification 


1 1 


of  the  submatrix  Fj  by  the  outer  product  aia^/<l£  to  give  *?£, 
which  is  the  submatrix  remaining  to  be  factored.  Thus,  the 
structure  of  is  different  froa  F£  due  to  a*aj/d£, 
a^a^AL  creates  new  fill-ins  if  (ap^fa^).  *  0  but  f^iVj  = 
0  where  faj^  is  a  tth  element  of  rector  a£  and  is 
the  scalar  eleaent  in  row  k  and  column  1  if 

It  can  be  seen  from  the  above  derivation  that  the 
matrices  D<k*  and  correspond  in  their  patterns  of 
nonzeros,  and  if  the  normalization  step  of  standard  Gaussian 
eliaination  is  replaced  by  "divide  row  \  by  (<?<£>)*/*,"  ’■her 
the  upper  triangular  matrix  produced  is  actually  T. Under 
the  assumption  of  noncancel lation  of  terms,  the  pattern  o* 
nonzeros  in  I?"  and  the  uoper  triangular  portion  of  n<m>  will 
be  the  same. 

3.  22  Factorization 


For  completeness  let  us  also  consider  OF  orthogonal 
factorization.  Orthogonal  factorization  methods  compute  thf» 
triangular  Cholesfcy  factor  L  directly  from  A  rather  than 
AAT  and  thus  avoid  explicit  formation  o*  the  cross  product 
used  in  basic  Gaussian  elimination  and  rholestv 
factorization.  Tn  OF  factorization,  an  orthogonal  matrix  o 
of  order  n  is  computed  which  reduces  A  and  c  to  the  form 


T  T 
0  A 


I 

0 


T  1  g  1 
and  o  c  =  !  f, 
I  r  ! 


where  g  is  a  vector  o*  dimension  m,  and  r  is  a  vector  of 
dimension  m-n. 

Then,  Dz  =  fc*  is  eouivalent.  to 
T 

AA  z  =  He, 

T  T  T 

=*>  *00  A  Z  *  APO  C, 

1? 


A 


'.{mJC* 


•  «'•>  A  A  _*•  ,** 


,*A "  ' 


% 


-->  [i  oi  |  i  j  i  ■  tm  i  j 


m 


==>  IL  T  -  La, 


==>  and  L  z  -  g. 


Th us,  after  comnutina  L  and  q  using  the  orthogonal  aa+ri* 
0,  we  need  only  solve  for  z  by  backward  substitution. 


0.  GRAPH  THEORETIC  HODEL 

Consider  the  two  matrices  below  where  x  indicates  a 
nonzero.  If  we  nerforn  Gaussian  elimination  on  the  two 
■atrices,  we  get  the  upper  trianaular  forms  shown. 
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gnder  assumotion  of  noncancellation  of  terms,  the  cholesVv 
factor  of  (1)  is  100w.  dense.  If  the  first  and  last  columns 
of  (1)  are  interchanged,  and  the  ^irst  and  last  rows  of  fl> 
are  interchanaed,  then  Gaussian  elimination  will  produce  no 
fill-in  at  all.  Felow,  a  graph  theoretic  model  is  described 


£Ref.  3"]  which  helps  clarify  the  process  of  creating  fill- 

0 

ins  ani  how  this  can  be  modified  by  reordering. 

Jin  undirected  graph  G={V,F)  consists  of  a  finite  set  of 
vertices  7  together  with  a  set  ft  of  edges,  where  an  edoe  is 
an  anordered  pair  of  vertices  (n,v)  such  that  n  *  v.  An 
undirected  graph  nay  be  associated  with  anv  symmetric  matrix 
n  of  order  a.  The  "adjacency  graph"  of  0  is  a  graph  G=(V,*j 
such  that  Vs (v,  .  ,vflvJ  and  (v£,vp  €.  F  if  and  only  if 
dij  *  0  for  all  i  <  1.  The  adjacency  graph  G<fc>  associated 
with  the  active  snbnatrix  of  is  called  an  "elimination 

graph."  Mote  that  an  adjacency  graph  G  associated  wi*h  o  is 
unaffected  by  any  synnetric  reordering  of  the  rows  and 
co limns  n. 

If  (u,v)  is  an  edge,  .a  and  v  in  5  are  "adjacent.”  vor 
S  CL  f *  the  adjacent  set.  of  S,  denoted  bv  Adj{S),  is  the  se* 
of  all  vertices  not  in  S  which  are  adjacent  to  vertices  in 
S.  That  is. 


Adj(SJ  =  (a  £  *-S  |  (a,v)  ^  F  for  some  v  £  S* 


where  V-S  is  the  set  of  all  vertices  of  v  which  are  not  in 

S. 

A  sobgraph  «•  =  (T«,EM  of  G  =  fV,F>  is  a  oraph  which 
consists  of  some  or  all  vertices  of  G  and  some  of  »-he  ed<res 
of  G z  V’  C  v,F*  C  F.  A  snboranh  is  a  clione  if  every  pair 
of  vertices  in  a  snbgraoh  is  adjacent. 

let  ns  consider  a  symmetric  soarse  matrix  and  its  asso¬ 
ciated  undirected  oraph.  The  matrix  is  of  order  F. 
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If  «•  perform  Gaussian  elimination  at  column  1,  we  obtain 

D<»l. 
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12395678 
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I 
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1 


where  new  fill-ins  are  noted  by  circles  and  -  corresponds  to 
a  nonxero  in  the  nonactiwe  part  o*  D<k>.  "ill-in  in  nt»> 
corresponds  to  edges  of  G<*>  which  were  not  in  gcoi  .  g<*> 
is  created  from  G<o>  by  deleting  V|  and  all  incident  edges 
and  by  then  connecting  all  vertices  which  were  ad  decent  to 
V|  and  not  alreadv  connected,  '•'his  is  formalized  below. 

let  G<k»  be  an  elimination  graph.  If  vertex 
v{  corresponding  to  row  i  and  column  i  is  next  selected  tor 
elimination,  then  the  new  fill-in  created  will  be  equal  to 
the  nunber  of  edges  in  the  complement  of  the  graph  induced 
from  G<k»  by  Adj(v|).  A  graph  G'fV*,*9)  is  "induced"  from 
G  (V, B)  by  V*  S.V  it  =  "-f(u,v)  £  x  s.t.  u  £  7»  or 
v  G * '  *  (7,***)  is  the  "complement"  of  G  i*  t"  * 


f(u,v)  s.t.  a,  v  6  7,  u tv  }  -  5. 

The  next  elimination  step  is  shown  below  *or 
completeness. 
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The  notion  of  "reachable  sets"  Is  useful  for  the  stu*v 
of  Gaussian  elimination  {George  and  fiu,  14V8a) .  Given  a 
graph  and  a  subset  5>  of  vertices,  if  u  and  v  are  two 
distinct  vertices  which  do  not  belong  to  ?,  we  sav  that  v  is 


reachable  froe  u  through  S  when  a  and  ▼  are  connected  by  a 
path  which  is  either  of  length  1  (u  and  v  are  adiacent)  or 
is  composed  entirely  of  vertices  vhich  belong  to  S.  Riven  S 
and  a  £  S,  the  reachable  set  Peach  fa,  S)  of  a  throagh  S  is 
defined  to  be  the  set  of  all  vertices  vhich  are  reachable 
froe  a  throagh  S.  When  S  is  empty,  or  vhen  a  does  not 
belong  to  Adj(S),  then  Peach  (a,  S)  =  Idjfu). 

A  tree  is  an  nndirected  acyclic  graph  vhich  is  connected 
and  has  no  cycles.  A  aatrix  aay  have  a  graph  vhich  is  a 
tree.  If  the  tree  is  labelled  in  increasino  order  of  vertex 
degree  and  the  aatrix  is  pernated  accordingly,  then  Ranssian 
elimination  vith  diagonal  pivoting  can  be  performed  vithou* 
any  fill-in  being  prodaced.  When  the  graph  of  a  matrix  is 
not  a  tree,  then  it  is  possible  to  find  a  tree  partitioning 
of  the  graph,  i.e. ,  a  partitioning  sach  that  the  "quotient 
graph"  is  a  tree,  let  the  set  of  vertices  7  of  a  graoh  R  Ke 
partitioned  into  the  disdoint  subsets  R|  ,  mhe 
quotient  graph  has  the  t  sets  Rj  as  composite  vertices,  and 
its  edges  are  defined  by  the  condition  that  fT^.nj)  is  an 
edge  if  and  only  if  tvo  vertices  a  and  v  *  q j  ‘  can  be 
found  such  that  a  and  v  are  ad-Jacent  in  R.  When  the 
quotient  granh  is  a  tree,  it  is  called  a  "quotient,"  and  the 
corresponding  pa rt it ion inn  a  "tree  partitioning." 

B.  CtfBBBWT  BBTHODS 

A  number  of  reordering  methods  are  known  vhich  dramati¬ 
cally  reduce  the  ^ill-in  resulting  *rom  ^actorixation. 
These  include  the  minimum  degree  algorithm,  various  dissec¬ 
tion  schemes,  and  various  bandwidth  or  profile  orderiro 
schemes. 

Band  and  profile  methods  regard  the  ^ill-in  in  a  olobai 
manner,  and  confine  it  to  certain  areas,  i.e.,  the  Mnd  or 
profile  of  the  matrix. 


The  dissection  ordering  nethod  is  a  nethod  for  systemat¬ 
ical!?  partitioning  the  graph  associated  with  a  natrir  using 
separators.  In  the  dissection  method,  the  fill-in  caused  bv 
Gaussian  elimination  with  diagonal  pivoting  is  confined  to 
nested  partitioned  areas  of  the  matrix. 

It  is  difficult  to  compare  methods  because  the  effi¬ 
ciency  of  a  nethod  denends  on  the  problem  or  class  of 
problems  for  which  it  is  being  used.  Thus,  we  will  onlv  be 
concerned  with  algorithms  which  do  not  exploit  soecial 
structure  except  implicitly.  The  nethod  which  is  currently 
accepted  as  best  for  "unstructured"  problems  is  the  "minimum 
degree  algorithm"  of  George  and  Liu  [Bef.  4%  This  thesis 
tests  for  efficiency  the  George  and  T.iu  algorithm  and 
several  other  algorithms  includino  variants  of  the  Georoe 
and  tin  algorithm. 


P.  TEST  PROBLEMS 


The  algorithms  described  in  this  thesis  are  tested  on 
sixteen  linear  programnino  and  mixed  integer  programming 
problems.  These  problems  consist  of  fourteen  real-world 
problems  (MR,  coal,  cups,  BOS,  rt"C,  po&y,  jcjv*>, 

PHST0B1,  PAD,  »TBS,  °TLOT,  S^PfT.,  TIG**)  and  two  artiricial 
problems  (STErwPPI,  STPTWFP2).  The  dimensions  of  these 
problems  are  shown  in  fable  T. 

All  programs  are  written  in  "Oi>T?*1d,  compiled  under 
POPTFAN  H-Pxtended  {Opt  2)  and  run  on  an  I"'!  under 
PB/CBS.  ComDutation  times  presented  in  the  following  chap¬ 
ters  are  accurate  to  the  number  of  decimal  places  shown. 
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II.  IfS  BIBIB  (TH  OBSBgg  A1GORITWB 
1.  .  IWTPO  DffCTIO  B 

In  this  chapter  we  discuss  the  minimum  degree  algorithm, 
with  details  of  the  implementation  by  George  and  liu  rw<»f. 
41.  This  algorithm  is  the  most  commonly  used  ordering  algo- 
riths  which  does  not  exnlicitly  exploit  special  matrix 
structures..  It  is  used  here  as  a  standard  for  comparison 
against  the  algorithms  given  in  chanter  III. 

The  main  idea  of  the  minimum  degree  algorithm  is  that 
when  we  perform  Gaussian  elimination  we  choose  the  vertex  of 
minimum  degree  from  the  elimination  graph  at  every  sten. 
This  myopically  minimizes  the  total  fill-in  versus  the  new 
fill-in.  If  there  are  several  minimum  decree  vertices  at  a 
particular  step,  ties  are  broken  arbitrarily. 


B.  BhSIC  ALGOPITHB 

algorithm  :  The  Basic  Binimum  Degree  Algorithm. 


Input  ;  elimination  graph  associated  with  a 

positive  definite  symmetric  matrix. 

Output.  :  A  permutation  vector  PE»»!1(  >  for  the 
vertices  of  G*°> 

Step  1.  (Initialization) 

Set  i  *  1. 

Sten  2.  (Select  minimum  degree  vertex) 

(i- 1) 

Tn  the  elimination  graph  G  *  ff  ,  * 

i-1  i-1 


select  a  vertex  v  6  v  which  is  of  minimum 

►  i-1 


degree. 


in 


Step  3.  (Transform  *he  graph) 


(i) 

Fora  the  new  eliaination  graph  G  =  (7^,F.) 


fi-1) 

by  eliminating  the  vertex  ▼  from  G  , 


where  all  vertices  adjacent  v  in  G 


(i-D 


are  connected  to  form  a  clique. 

Set  PERM(i)  =  *. 

Step  4.  (Loop  or  stop) 

Set  i  *  i  ♦  1. 

Tf  i  >  17) ,  go  to  sten  5. 
Otherwise,  go  to  step  2. 

Step  5.  (Output) 

For  j  =  1  to  |7f,  print  PEPE(i). 


Pnd  of  The  Basic  dinimun  Degree  Algorithm. 


C.  BWH11ICFD  M.GOFTTHH 

We  can  enhance  the  minimum  degree  algorithm  by  using  »-he 
concept  of  "indistinguishable”  vertices  by  George  and 
tiu(1981).  Two  unlabelled  vertices  u,  v  fe  v-S  are  said  to 
be  indistinouishable  if  Peach  (u,S)  U  fu'  *  Peach  (v,S)  (J  fv*. 
At  any  step  t,  if  we  find  Indistinguishable  vertices  they 
can  be  labelled  next  without  any  minimum  degree  search. 

Algorithm  t  The  Enhanced  Einimum  Decree  Algorithm. 

Input  ?  Elimination  graph  G<®>  associated  with  a 
positive  definite  symmetric  matrix. 

Outnut  :  A  permutation  vector  nEPBf  )  for  the 
vertices  of  G<°» 

2D 


•  »  > 


Step  1.  (Initialization) 

Set  i  =  1. 

Step  2.  (Select  minimum  decree  vertex) 

Tn  the  eliaination  graoh  g**  ^  =  (V  ,  v  >. 

-  i- 1  i-1 

select  a  vertex  v  fi  v  which  is  of  minimum 

e  1"  i 

degree. 

Step  3.  (Find  the  indistinguishable  vertices) 

Find  the  vertices  which  are  indistinguishable 
frow  v.  let  T  he  the  set  of  v  and  all  vertices 
indistinguishable  from  v. 

Step  4.  (Flinination) 

For  each  ver+ex  u  €  Y?  Fiininate  u  in 

to  obtain  fftl\  set  PFFSfi)  *  nr  and 
set  l  -  i  ♦  1. 

Step  5.  (Loop  or  stop) 

If  O  )▼(,  go  to  step  6. 

Otherwise  go  to  step  2. 

Step  6.  (Output) 

’or  1  -  1  to  |vj,  print  t»wv»(1>. 

Fnd  of  The  Enhanced  Sinimum  Degree  Algorithm. 


0.  IHPlBSFHThTIOW 

The  implementation  of  th®  minimum  degree  algorithm  uses 
the  basic  ideas  of  elimination  graphs,  reachable  sets, 
indistinguishable  vertices  and  is  further  enhanced  through  a 
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A 


V? 


quotient  graph  partitioning  of  the  initial  eliaination 
graph. 

The  graph  G  -  (V,S)  is  stored  using  the  adjacency  array 
pair  (XADJ, ROJllCY)  in  which  the  anount  of  storage  need  is 
0(JT|)  and  Off*!)*  respectively.  The  anount  of  storage  of 
the  reaaining  arrays  is  0{|VO»  RDJNCY  gives  the  adjacent- 
sets  Xd;j(v)  in  contignons  storage  locations  for  all  v. 
XXDJ (v)  gives  the  location  of  the  first  element  Rdj(v)  in 
kDJmCY.  The  resnlting  ordering  array  pair  is  pvny.  The 
algoritha  requires  the  determination  of  reachable  sets  for 
degree  update. 

Vertices  identified  as  indistinguishable  are  merged 
together  to  fora  a  superverter.  They  are  treated  as  on® 
vertex  as  the  algoritha  continues.  The  current  degrees 
the  vertices  in  the  elimination  oranh  are  kept  in  the  arrgv 
DFG. 

Rn  initial  partitioning  of  the  graph  is  made  to  create  a 
quotient  graph  which  is  a  tree,  ’’•hen,  vertices  of  the  elim¬ 
ination  graph  are  selected  only  from  within  subsets  o* 
vertices  making  up  a  comoosite  decree- 1  verterr  of  ♦he  ♦h® 
quotient  graph.  tn  this  way,  fill-in  can  be  guaranteed  not 
to  extend  outside  of  the  quotient  graph.  This  technique 
helps  keep  fill-in  low  globally. 


Ill*  IIS  HTWIHOB  FltL-IW  HtGOBITHH  MID  T1BI1UITS 
ft.  I»TI»»tKmO!f 

In  this  chapter  we  introduce  the  minimum  *ill-in 
algorithm.  In  the  minimum  degree  algorithm,  *he  main 
concept  is  to  select  that  row  and  column  which  wields  the 
least  total  fill-in  in  the  next  active  subaatrix,  while  in 
the  minimum  fill-in  algorithm  the  central  idea  is  to  select- 
that  row  and  column  which  introduces  the  least  new  fill-in. 

Suppose  {▼,,  v^,....,  w^_| >  have  been  labelled.  It  is  a 
simple  matter,  conceptually,  to  comoute  the  new  fill-in 
which  would  be  Introduced  by  selecting  any  vertex  as  Hie 
next  vertex  to  be  labelled  as  V£  *  fill(v£)  is  the  number  oe 
vertices  vj,  Tjc.€  'dj(v^)  such  that  vj  is  not  adjacent  to 
v^.  In  practice,  t-his  number  can  be  comouted  easily  i  f 
*dj(v)  is  kept  as  a  linked  list  for  all  unlabelled  vertices 
v.  Below,  the  minimum  cill-in  algorithm  is  described 
followed  by  more  detail  on  its  implementation. 

B.  THE  HIBIHBB  EIH-IW  IIGOPITHB 
Algorithm  :  '’’he  dinimum  fill-in  Algorithm. 

Input  :  Elimination  graph  q<o>  associated  with  a 
positive  definite  symmetric  matrix. 

Output?  A  permutation  vector  Pf«d(  )  for  th®  vertices 
of  o<o> 

Step  1.  (Initialization) 

Set  i  *  1. 

Compute  *ILT,tw(v)  for  all  v  6  v*  the  amount 
of  new  fill-in  which  would  be  created  i^  v  is. 
eliminated. 


Step  2 

Step  3 


Step  4 

Step  5 

Step 


(Select  ainiana  fill-in  vertex) 


Tn  the  eliaination  praph  G 


(i-11 


=  (V.  , 


i-r  i-1 


select  a  vertex  v^€  7^  ^  sach  that  ^TltTViv) 


is  ainimna. 

(Transfora  the  grap*) 


(i) 


Pora  the  ne»  eliaination  graoh  o  =  (7  rP  ) 

l  i 


by  eliainating  the  vertex  v^  froa  G 


fi-1) 


(i- 11 

where  all  vertices  a^lacent  «-o  v  in  5 

fc 

are  connected  to  fom  a  clicrne. 

Set  PPPUfi)  =  k. 

.  (Pirate  VTL1TM  array) 

Tjnda*e  PTT.LIP(V)  for  all  v  £  7  . 

i 

.  (loop  or  stop) 

Set  i  =  i*1. 

If  i  >  17|,  go  to  step  f . 

Otherwise  go  *o  step  2. 

>.  (Oatpat) 

Por  j  =  1  to  1 7 f r  print  °PP"!(1). 


Pni  of  The  fininaa  Pill-in  Alaoritha 


C.  IHPLEHEWT1TI01I  OF  THE  HIWHHH  FILI-IH  ItSOHXTHH 

The  graph  6  =  (T,E)  is  stored  using  the  linked  adjacency 
list  arrays  (XRDJ#IDJWCY,I.INK1)  f?ef.  where  the  pointer 
XIOJ  (i)  starts  the  adjacency  list  for  wertex  ir  with  kniticy 
containing  a  neighbor  of  vertex  i  and  tT*K  containina  the 
pointer  to  the  location  of  the  next  neighbor  of  wertex  i. 
The  number  of  vertices  in  v  is  given  by  HPPN5.  This  *ata 
structure  requires  more  space  than  the  Georoe  and  tin 
algorithm;  but,  it  requires  space  proportional  to  *ha* 
necessary  for  carrving  oat  one  of  the  Gaussian  elimination 
techniques.  Thus,  this  is  not  really  a  limitation  if 
storage  is  reused. 

The  purpose  of  the  above  algorithm  is  to  find  *h° 
minimum  fill-in  vertex  for  a  general  graph.  It  operates  on 
the  input  graph  as  given  by  HTOWS  and  the  linked  adjacencv 
list  structure  (XHPJ,  AT>  TXCT#IIPT 1)  ,  and  returns  the  orderin  '! 
in  vector  PPBH. 

The  arrav  PIT  IIP  (v)  stores  the  number  of  new  fill-ins  if 
vertex  u  is  eliminated  at  the  next  steo.  PTT  T.TU  fv'  is 
updated  after  each  elimination  step.  This  need  be  done  onlv 
for  those  vertices  which  are  a  distance  o*  two  edoes  or  l»ss 
from  u  before  the  elimination  steo.  mhe  PTT  Liu  value  fo- 
vertices  that  have  been  eliminated  is  set  neorative. 

Initial  testing  of  the  minima*  fill-in  algorithm  has 
shown  that  it  is  much  too  time-consuminrr.  Consequent lv,  * 
restricted  version  has  been  developed  which  only  searches 
among  the  minimum  degree  vertices  for  the  minimum  *ill-ir 
vertex.  This  is  really  a  hybrid  of  the  minimum  decree  alao- 
rithm  and  the  minimum  fill-in  alcorithm  and  is  called  *he 
"min  decree  /  rain  fill-in  algorithm". 
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0.  HIV  DS6V1H  /  HIV  FHI-IV  AIGOHITHH 


Algorithm  s  The  Hin  Degree  /  Hin  Fill-in  Algorithm. 


Input  ? 
Output: 
Step  1. 
Step  2. 

Step  3. 


Elimination  graph  G<°>  associated  with  a 
positive  definite  symmetric  matrix. 

A  permutation  rector  PSFHf  )  for  the  vertices 
of  G<°> 

(Initialization) 

Set  i  =  1  and  ad  =  0. 

(lazy  update  of  fill-in  array  FIT.! I!*) 

Find  minimum  degree  nindeg. 

If  ad  F  nindeg,  set  ad  *  nindeg  and  compute 
FltLIV (v)  for  all  vertices  v  with 
deg(v)  =  ad. 

Otherwise  go  to  step  3. 

(Select  ion) 

(i- 1) 

In  the  elimination  graph  G  =  fv  ,  y  > , 

i- 1  i- 1 


select  a  vertex  v  6  V  which  is  of  mininun 

k  i-1 


fill-in  among  the  minimum  degree  vertices. 


Step  4.  (Transform  the  graph) 


(i) 

Form  the  new  elimination  oraph  0  =  (V  ) 

i  i 


(i-1) 

bv  eliminating  the  vertex  v  from  G  , 

k 


where  all  vertices  adjacent  to  v  in  o 

K 


ri-i) 


are  connected  to  form  a  eliaue. 
Set  PFFH(i)  =  V. 

Step  5.  (loon  or  stop) 

Set  i  =  i*1. 


If  i  >  | ?f ,  go  to  step  6. 
otherwise  go  to  step  2. 

Step  6.  (output) 

For  j  =  1  to  m,  print  PCTB(I). 
Bad  of  The  Bin  Degree  /  Bin  Fill-in  Algorithm. 


E.  OTHE*  FIEIIETS 

The  linked  adiacency  list  stnctnre  used  in  the  win 
degree  /  nin  fill-in  algorithm  makes  it  easy  to  try  other 
ordering  methods.  Three  additional  methods  will  be  tested  ? 
a  sinple  minimum-deqree  algorithm,  an  initial  minimum  degree 
algorithm,  and  an  initial  ordering  algorithm. 

The  "sinple  mininnn-degree  algorithm"  is  essentially  the 
George  and  tin  algorithm  hat  with  a  different  data  strnctnrs 
and  without  the  reachable  set  and  gaotient  graph 
enhancements.  The  "initial  minimum  decree  algorithm" 
computes  the  degrees  of  the  vertices  in  G<°>  and  selects 
vertices  in  increasing  order  of  these  degrees.  Bo  uodate  of 
the  initial  degrees  is  ever  made.  The  "initial  orderino 
algorithm"  lust  takes  the  vertices  of  G<®>  in  the  order  in 
which  they  are  given,  i.e. ,  in  the  order  in  which  the  rows 
of  the  k  matrix  are  given.  This  is  a  rather  trivial  algo¬ 
rithm  but  the  time  to  create  the  compact  data  structure  ^or 
sparse  Gaussian  elimination,  inclnded  for  the  other  algo¬ 
rithms  is  shown. 


w 


v.y>.v.v.i 


rarest 


V.v.s>V.>v.V 


If.  COHPOT&TIOWHI  ggPgEIgWCg  HP  PITglCtJLTTgS 

To  solve  the  sixteen  test  problems,  we  hare  use^  six 
methods  to  see  which  method  is  lost  effective.  The  methods 
ares 

Method  1.  The  minimum  decree  algorithm  of  Georoe  and 
Lin. 

Method  2.  The  minimus  fill-in  algorithm. 

Method  3.  The  min  degree  /  ain  fill-in  algorithm. 

Method  4.  The  simole  minimum-degree  algorithm. 

Method  5.  The  initial  minimum  degree  algorithm. 

Method  6.  Initial  ordering  algorithm. 

Pesults  for  method  2  are  not  included  because  initial 
run  times  were  excessive  even  for  the  smallest  problems.  As 
shown  in  Table  III,  two  problems  (JChb,  were  not  solved 

due  to  lack  of  computer  memory.  Time  is  rprj  seconds  neces¬ 
sary  to  obtain  the  ordering  and  to  then  create 
compressed  data  structure  used  by  George  and  Liu. 

The  minimum  degree  ordering  algorithm  of  George  and  *  in 
is  best.  the  simple  minimum  degree  algorithm  with  linked 
data  structures  is  almost  as  good  -  it  is  sometimes  faster, 
and  it  is  sometimes  slower.  Put,  new  fill-ins  created  are 
always  more  than  that  of  the  minimum  decree  alcorithm  of 
George  and  Lin.  The  quotient  graph  enhancement  of  the 
George  and  Lin  alcorithm  is  probably  resoonsible  tor  its 
snperior  performance.  Mthouoh  storage  requirements  usinc 
the  linked  list  data  structure  are  creater,  this  onlv  seemed 
to  be  a  problem  when  the  fill-in  became  very  large.  this 
structure  gives  a  very  flexible  frameworv  for  t.rvinc 
different  heuristics. 


RESULTS  OF  CBDEBING  AIGOBITHNS 


<tf 1 

(Nororf 

o»- 

CNm»OtNO 

\o  a  i 

•  •••••  1 

«  1 1 

•  •••*« 

•H  1 

oooooo 

oo 

oooooo 

AH  1 

O 

J3 

■MH » 

coeoomo? 

oco 

oco'-oir»vo 

4»r-(  | 

i 

•  •  i 

•  ••••• 

■3'H  1 

om 

«"(N(flOO 

U4  | 

rncN 

vo^ottno-oi 

0)  1 

04O|»04r»(,"t 

O 

a— 

o 

f)  3  i 

*«•••• 

1  •  t 1 

•  1 

•  •  i 

• 

-t  1 
OH  1 

O 

»*ONO»,'0 

a— 

fn 

\Of- 

o 

i 

n-oar'<u» 

00H 

»-o 

o> 

0)H  | 

•  ••••• 

1  •  •  1 

•  i 

*  •  1 

• 

| 

t-f-orveno 

oo 

o 

m 

*■  1 

04  Ol 

o» 

04  vO 

— 

y  i 

'XJrsjUTCNO^ 

OO' 

:»04CT>  vO 

jt  a  | 

•  •  t  •  •  1 

1  •  •  l 

1  •  •  •  1  • 

-H  1 
OH  | 

O 

G 

rOi'NOr" 

oo 

GOO  o 

•4-H  1 

r*^y 

cutflO  O' 

I 

•  t  •  •  •  1 

i  •  t  1 

1  I  •  •  1  • 

*3H  1 

;Nn^no 

om 

'—04'—  04 

1 

cm*-*  m 

rn 

!N*-vO  «- 

y  i 

'^-Oinrf-OO  — 

iO 

o  a  i 

••III 

1  •  1 

• 

-t  i 

no»“'-GP'  oca 

o 

*— 

T3H  1 

*~  »“-'4 

a 

G 

■M-H  1 

■cou>nm!N  o-o 

<UH  1 

•••••»i  ••iii 

»  •  1 

• 

5E-*  1 

oio^*  -ro  >4  om 

O' 

'N 

U.  1 

oj>—  rn  o 

tn 

a»  i 

v-^T^COT-O 

o 

—  a  i 

t  •  • 

1  >««••• 

H  1 

mOir.o*-*- 

oo 

vNOI 0-0040 

■QH  | 

V-  fN 

U 

Z>H  | 

rT.osr  .o  -OfN 

1/1^ 

aiH  i 

•  ••••« 

•  •  • 

1  •••••• 

K  H  | 

•— OOJT— r-O 

cna>Lns3c>o 

U,  1 

04 

UT-O 

Gj©  1 

-,o  /v"Mr*  rrm 

o*---^r\|rnm 

a  a  i 

•  »••••  1  •  i  i 

-W--4  | 

rsio^v-r-v-  op* 

BH  1 

N3-  r4 

CM^<N 

(O  | 

*-.N 

a  i 

'Cti 

y  i 

za 

WW 

-H  | 

MO 

HM2!hU; 

-Q  1 

mOWMMW 

O  1 

CS  Cfl  <  CU  M  as  «*  H  t/>  Q  W  -4  W  U4  W  rj 

M  1 

H3yajy’j«44MHHHH*' 

Bj  I  43  JUbLUt^ZCtaDiOj^l/l^M 


-  leans  not  solvable  with  2  meqjtytes  of  ueaocy 
“fill"  aeans  new  fill-in. 


TABLE  III 
FILL-IB  RESULTS 


Problem 

Initial 

Pill-in 

Pinal 
Bo  Order 

fill-in 

Best  Order 

Bew 

Bo  order 

*ill-in 

Best  order 

AIR 

20.8 

98.6 

22.  1 

77.8 

1.3 

BOS 

•>5.7 

79.5 

76.4 

23.8 

20.6 

COAL 

26.  1 

38.0 

34.5 

11.9 

9.  a 

CUPS 

1.3 

8.  8 

2.9 

7.5 

1.6 

ELEC 

2.0 

15.0 

3.8 

13.9 

1.6 

POAN 

1.8 

2.2 

2.0 

0.4 

0.2 

.TCAP 

— 

— 

— 

— 

— 

BETTTBG 

3.8 

41.8 

4.3 

37.9 

0.5 

PASTOP1 

1.7 

27.4 

5.7 

75.8 

4.  1 

PAD 

— 

— 

— 

- 

— 

PTES 

1.9 

69.8 

11.0 

68.  0 

9.  1 

PILOT 

2.5 

44.3 

11.9 

41.8 

9.4 

STEEL 

3.0 

25.1 

8.7 

22.1 

5.6 

STETBEP1 

31.0 

87.2 

87.4 

56.2 

56.4 

STEIBEE2 

19.  1 

89.8 

89.4 

79.7 

69.3 

TIGE8 

14.7 

35.3 

24.6 

20.6 

Q.O 

The  ain  degree  /  ain  ^ill-in  algorithm  performed  noorlr. 
It  is  always  slower,  and  the  new  fill-ins  created  a^e  always 
aore  than  that  of  the  Reoroe  and  Lin  aloori^hm.  It.  canno* 
be  recommended  in  any  case  and  it  is  nnclear  wh*  i* 
performed  so  poorly. 

Final  fill-in  for  STETBE®1  and  SEETHE*?  in  "able  IT*  is 
high,  bnt  these  are  artificial  combinatoric  problems.  Pn9 
is  a  small  real-world  problem  with  a  best  final  fill-in  o* 
16.4*  which  is  mnch  higher  than  the  others.  The  probable 
reason  for  this  is  that  BUS  is  a  set  nart.itionino  orobtem. 
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TIGFF  with  2*. 65  final  fill-in  is  also  a  set  partitioning 
problem.  This  indicates  that  the  protective  algorithm  might 
not  work  very  veil  on  aany  set  partitioning  and  set.  covering 
problems  since  these  seem  to  have  a  lot  of  intersections 
along  rows.  The  lowest  final  fill-in  found  was  2.05  on 
FOAH.  Six  of  the  problems  had  less  than  lO.o*  final  fill-in. 


An  interesting  comparison  can  be  made  between  the  initial 
fill-in  and  the  original  densities  of  these  problems.  Th® 
initial  fill-in  can  be  compared  to  the  density  of  matrix  A 
which  is  about  the  same  as  the  density  for  a  basis  in  th"* 
simplex  algorithm.  This  gives  a  idea  of  how  hard  creating  L 
is  compared  to  creating  for  the  simplex  algorithm.  Th® 
initial  fill-in  of  FPS,  STFT**F1  and  ST*T wye?  was  greater 
than  IS. 15  and  they  had  densities  greater  than  6.7*.  nut, 
»IF,  COM  and  TTGF»  had  more  than  1tt.7«  initial  fill-in, 
while  these  had  densities  of  only  1.2*,  1.75  and  ft.1* 
respectively.  The  simplex  algorithm  might  have  a  relatively 
greater  advantage  in  these  last  cases. 


T.  COWCLPSIOHS  1WD  BgCOHHUtPiTTOWS 


This  thesis  examined  various  heuristic  algorithms  for 
ordering  symmetric  linear  systems  of  equations  to  minimize 
fill-in-  The  algorithms  were  tested  on  problems  derived 
from  linear  programming  problems  in  the  manner  of  ♦•he 
projective  algorithm  for  linear  programming. 

The  minimum  degree  algorithm  of  George  an*  I im  is  the 
most  effective  of  six  tested  methods-  Solution  tines  for 
this  algorithm  are  sometimes  better  than  and  sometimes  worse 
than  the  tines  using  a  simple  minimum-degree  algorithm,  but 
new  fill-ins  created  are  always  less-  Art  a  result  o* 
storage  limitations,  the  simple  minimum  degree  algorithm 
failed  to  solve  two  orohlem3  which  the  George  and  tin  uiuo- 
rithm  solved-  This  should  not  be  a  problem  in  oractiee  if 
storage  is  reused. 

The  minimum  fill-in  algorithm  and  the  min  degree  /  min 
fill-in  algorithm  used  here  do  not  perform  as  well  as  might 
be  hoped.  The  minimum  fill-in  algorithm  is  too  slow  to  use 
in  most  cases.  Both  algorithms  involve  more  work  than  the 
minimum  degree  algorithm  of  George  and  T.iu  and  the  ordering 
produced  is  rarely  much  better  than  the  one  produced  tv  the 
minimum  degree  algorithm  of  George  and  Lsu. 

The  quotient  graph  enhancement  of  th«*  George  and  tiu 
algorithm  is  probably  resnonsibie  for  its  suoerior 
performance.  Thus,  future  research  should  examine  the  use 
of  the  quotient  graph  and  the  min  degree  /  min  fill-in  ideas 
since  it  nay  still  he  possible  to  sionificantlv  improve  uoon 
the  new  fill-ins  obtained  without  sacrificing  much  computa¬ 
tional  soeed.  Different  heuristics  for  creating  quotient 
graphs  should  be  exatined,  too- 


LIST  OF  FFFMFWCFS 


Narendra  K.  Karnarkar, 
11216-840126-04,  ATST  Sell 


technical 

Eabbfatotes 


Nesorandas. 

rny r. — 


"*1 


Brown.  Pichard  0.  Ncbride  and  4.  Fevin  Wood, 
Nabeoded  Generalized  Networks  froa  linear 
PFSBIf'i?.  nath^watidal  Pf  Sdfawilrid,  7oT7 
3TT77E7T 


Seroio  Pissanetsky,  Sparse  Hatri*  ’’’echnology ,  Acadenic 
Press,  1984.  “  ” 


Gerald  G. 
Nxtractlnc 
Prddaafnn 
727 


Ulan  George  and  Joseph  4-9  tin.  Cowonter  solntion  o* 
Large  Sparse  Positive  Definite  SvsFFIS,  PFeirfTJJ^-^SlTT 

TT1577  m - 


Cartis  F. 
Nanerical 

ttht; - 


Gerald  and  "atrick  0 
Ana Ivsis,  Addison  - 


.  Wheatley,  Applied 
Wesley  "nblishiKirTTlTT 


Howard  Anton,  Fleae ntarv  T inear  Algebra.  John  Wiley  *“■ 
Sons,  1981.  ** 


Hichael  T.  Heath,  Nnsencal  Hethods  for  targe  Sparse 
Linear  Least.  Sgaares~PF6BIg]f3.  syiN  1.  ~cl.  CoWHHT. , 
7nrT~S.  ~Wd7~S7iiHB7~4ST=STT7  September  19*4. 


Alfred  7.  Aho,  John  f.  Hoocroft  and  Jet*rev 
The  Design  and  Analysis  of  Cowpnter 
TcIdiso!T-'W5?ley~«nMT?irt15d~ro«BSny~^74“ 


n.  Hilean, 
Aloori^hss , 


Lawrence  7.  Corwin  and  ®obert  ».  SzozarM. 
Haiti variable  Galenas .  *aroel  nekker,  Tnc.  Hew  Tork 
iiTd~Ha^?T7_TW8T; - 


David  G.  Luenberoer,  Linear  ana  nonlinear 
Addison-weslev  Pablishlfid  ToiTiSny7”"T995. 


Lnenberoer,  Linear  and  Nonlinear  Progr aswi r g. 


IRITT1L  DISTFIBOTIOR  LIST 


Copies 


1.  Defense  Technical  Information  Center 
Cameron  Station 

Alexandria#  Virginia  22304-6145 

2-  Library#  Code  0142 

Waval  Postgraduate  School 
Monterey,  California  93943-5100 

3.  Department  Chairman,  Code  55?s 
Department  of  Operations  Research 
Ifaval  Postgraduate  School 
Monterey,  California  03043-5100 

4.  Professor  Fevin  Food-  Code  55Pd 
Department  of  Operations  Pesearch 
naval  Postgraduate  School 
Monterey,  California  03043-5100 

5.  Professor  Gerald  0.  Brown,  Code  55Bv 
Department  of  Operations  Pesearch 
naval  Postgraduate  School 
Monterey,  California  03043-5100 

6.  Professor  Gordon  H.  Bradley,  Code  5?Px 
Departaent  of  Computer  Science 

naval  Postgraduate  School 
Honterey,  California  93043-5100 

7.  Professor  Pichard  Posenthal,  Code  55vi 
Department  of  Operations  Pesearch 
naval  Postgraduate  School 

nonterey,  California  93943-5100 

B.  Professor  Arthur  Schoentadt,  Code  537h 
Departaent  of  Mathematics 
naval  Postgraduate  School 
Honterey,  California  03043-5100 

9.  Professor  Glenn  Graves 
Graduate  School  of  Management 
University  of  California.  Los  Angeles 
Los  Angeles,  California  90024 

10.  Major  Young  Ho  Seo 

449  SeoJr-Nam  Dong,  Buk  Gu, 

In  Cheon,  Seoul,  Forea 


’j*  v*  •  •  •  •  •  • 


‘  vvv-:vyvyy.' 


12-85 


DTIC 


